1. Field of the Invention
The present invention relates to an image-taking apparatus which employs an image-pickup element to take image information, and more particularly, to an image-taking apparatus which performs fluctuation correction processing on a taken fisheye image to output the resulting perspective projection image.
2. Description of Related Art
Conventionally, image-taking operation in a wide-angle field of view with fisheye optical system has been performed in variable areas such as monitoring, remote operation, video conference, and endoscopies. The fisheye optical systems are characterized in that even an image in a super wide-angle field of view at an angle of field of more than 180 degrees can be projected in a finite area on a light-receiving surface of the image-pickup element through a special projection method, in other words, it is possible to take an image in a super wide-angle field of view which cannot be taken theoretically by a general image-taking apparatus for taking a perspective projection image. In the projection method specific to the fisheye optical system, however, the shape of a taken image is distorted to the extent that it looks significantly different from the original shape of the object when viewed by a human.
As an approach to solve the problem, WO 92/21208 (corresponding to U.S. Pat. No. 5,185,667 and Japanese Patent Laid-Open No. H06 (1994)-501585) has proposed an apparatus which transforms a taken fisheye image into a perspective projection image and displays the transformed image on a display. In Japanese Patent Laid-Open No. H06 (1994)-501585, panning, tilting, rotation, and magnification are set for an arbitrary image area in a hemispherical field of view taken by a fisheye lens, and coordinate transformation is made by using two-dimensional transformation mapping, X-MAP and Y-MAP, thereby making it possible to observe an image of all directions.
Japanese Patent Laid-Open No. H10 (1998)-233950 (corresponding to WO 98/27718) has proposed electronic zooming which utilizes the distribution of the density of image information in an image taken by a fisheye optical system. Specifically, the image taken by the fisheye optical system has the characteristic that it is more highly compressed in a portion closer to its periphery and is more greatly enlarged in a portion closer to the center. The characteristic is used to perform the electronic zooming without reducing resolution.
In addition, Japanese Patent Laid-Open No. H8 (1996)-307753 has proposed an apparatus which can display data corresponding to part of an image in a wide-angle field of view taken by a fisheye lens without distortion in accordance with the movement of the head of a viewer. Specifically, Embodiment 3 described in Japanese Patent Laid-Open No. H8 (1996)-307753 involves transformation processing for displaying an image taken by the fisheye lens with an equidistant projection method in a planar shape without distortion and compensation processing for image fluctuations due to vibrations of the image-taking apparatus. The compensation processing is performed when the amount of pixel shifts associated with the image fluctuations is equal to or smaller than a predetermined number of pixels (for example, two pixels).
In the apparatus of Japanese Patent Laid-Open No. H06 (1994)-501585, a field of view and a magnification are designated from an input means, a transformation expression is solved by microcomputer, and a transformation table is set for a two-dimensional transformation map to provide the panning, tilting, rotation, and zooming functions in an arbitrary area. Since the transformation expression must be solved each time a slight change occurs in the field of view or magnification, the problem of a significant load of operation on a microcomputer is presented.
In the apparatus of Japanese Patent Laid-Open No. H10 (1998)-233950, an electronically zoomed image with less deterioration of resolution is provided by using the density distribution of image information in a fisheye image. However, as the zoom magnification is greater, image fluctuations are caused by vibrations applied to the apparatus such as camera shake to result in the problem of deterioration of the resolution.
The apparatus of Japanese Patent Laid-Open No. H08 (1996)-307753 performs control such that a comparison between the detected pixel shift amount and the predetermined pixel number is used as a reference for performing the compensation processing for image fluctuations. However, the pixel shift amount due to video fluctuations is determined by a relationship between the vibration amount applied to the apparatus and the zoom ratio and is not determined by a single threshold value. In addition, the determination whether image fluctuations are caused by camera shake or by an intentional movement of the field of view such as panning and tilting cannot be made on the basis of a fixed threshold value, and there are large image fluctuations due to unintentional violent vibrations or intentional slight image fluctuations for a slow movement of a viewpoint.